The present invention relates to software development, and more specifically, to systems and methods for the integration of a content space with test planning and test generation.
In the domain of software development, use cases and user stories are used to represent an external or end-user view of a product. As such, testing activities such as function verification test (FVT) and system verification test (SVT) directly execute use cases and user stories as at least part of their testing. Typically, FVT and SVT test plans explicitly address use cases, and agile stories have introduced the notion of user story.
Basic inquiries regarding testing of a given product release start with use cases or user stories and the plans that are created to test subsets of these based on different criteria. For example, inquiries include whether the use case was changed in any way in the current release. If the use case was not changed in the current release, the testing is typically narrower to ensure the use case (or ‘function’) works as it did in a prior release. For new use cases, the test plan calls more extensive test cases, exploring both positive and negative conditions, boundary conditions, combinations of platforms and the like. For changed use cases, testing can be more efficiently focused on the changes, rather than testing the entire use case or use story.
Several inquiries typically remain such as but not limited to: 1) whether the use cases or user stories are complete; 2) how completeness is determined; 3) the criteria to determine completeness; 4) whether overlap exists among use cases and user stories (overlapping use cases tend to reduce testing efficiency and greatly complicate test planning and test execution.; 5) determination of gaps between use cases and user stories; 6) a determination of functional capabilities that exist in the product, which are not covered by use cases or user stories; 7) if use cases and user stories are the primary or key input to test planning and test case definition, how gaps are detected; 8) a determination of the quality of testing for each requirement; and 9) whether requirements traceability is sufficient to obtain inquiries via automation.
The above-described inquiries are currently recognized and various mitigations have become fairly typical industry ‘best practices’. For example, the FVT test leads will typically review the product design documentation to determine what function exists and help ensure the test plans are complete without being too redundant. In another example, development teams review FVT test plans and sign-off that they are ‘complete’ (e.g., address a domain of product functions without gaps). Both examples depend on numerous people writing and reading numerous documents, therefore having inherent costs and marginal effectiveness. The aforementioned and other mitigations suffer from numerous and well-known weaknesses that routinely impact software product development.
Currently there exists a lack of logically complete input to test planning and test case generation that can utilize further automated techniques to support test scaling and efficiencies for today's software products. More specifically, the input to test planning and test case generation needs to be automatable, complete under some reasonable definition, consist of discrete units which are guaranteed to be non-overlapping and with no gaps, and support automated traceability back to requirements.